Ideally, a pilot or other vehicle operator can see the external environment as they maneuver their vehicle. However, in some circumstances, a vehicle operator needs to operate his vehicle when visibility is poor. Sensors may be used to aid the vehicle operator in these conditions. For example, an aircraft may include infrared (IR) sensors and millimeter-wave (MMW) sensors, which may be used in conjunction with video cameras (typically operating at other wavelengths, including the visible band), to provide the pilot with a computer-generated view of the external environment. While helpful, the sensor data may be limited by field of view, range, the limited two-dimensional flat view, and operational response problems related to weather conditions or other obscurants.
Synthetic data from a database may also be used to provide a vision of the external environment to a vehicle operator. A synthetic image database includes previously obtained images of the terrain indexed by Global Positioning System (GPS) data, other navigation solution data, and/or other available datasets. A Synthetic Vision System (SVS) may construct a particular view of the world from the database based on the vehicle's location, map the information into a three-dimensional image, and display the constructed image to the vehicle operator.
The image shown to the vehicle operator may depict the terrain in the location of the vehicle, in a photorealistic display, similar to a three dimensional (3D) picture used in video games. Thus, the vehicle operator may be able to envision the terrain, including obstacles, without actually seeing the external environment. For example, a pilot using the SVS may see a 3D image of an approach path and runway during landing.
However, the SVS is limited to the datasets stored in the synthetic image database. The synthetic image database may not be complete or may include erroneous data points. Additionally, the synthetic datasets do not include dynamic variables. As a result, the SVS does not display images showing real-time wildlife, vehicles, or other changed conditions since the data in the database was obtained.
Previous attempts to fuse sensor and synthetic data to form an integrated image have failed. The difficulty is in positioning the two images so that the features in the images are aligned. If the two images are not aligned, the resulting image may actually confuse a vehicle operator instead of providing guidance.
Thus, it would be beneficial to fuse the sensor data with the synthetic data into a single integrated image. The integrated image may be provided to vehicle operators in low or no visibility conditions, allowing for safer vehicle operation in these poor conditions.